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Geopositional Accuracy Validation of EarthSat GeoCover™ Orthorectified Landsat 
ETM+ Imagery 

Scope of Work 

This report provides the results of two independent evaluations, an absolute and a relative 
assessment, of the geopositional accuracy of the Earth Satellite (EarthSat) Corporation s 
GeoCover™ orthorectified Landsat Enhanced Thematic Mapper Plus (ETMN imagery. 
This imagery was purchased through NASA's Earth Science Enterprise (ESE) Scientific 
Data Purchase (SDP) program via NASA contract NAS 13-02032. 

Background 


One of the techniques used to validate the geopositional accuracy of EarthSat s E1M 4 - 
imagery required the use of validated Landsat Thematic Mapper (TM) imagery. The TM 
imagery used in this assessment was purchased from EarthSat through the SDP via 
NASA contract NASI 3-98046. The geopositional accuracy of this TM imagery was 
independently validated by Earth Science Applications (ESA) Directorate personnel prior 
to this effort. 

The TM sensor onboard Landsat 4 and Landsat 5 is a high-spatial-resolution (30-meter 
ground sample distance (GSD)), multispectral system with seven bands in the visible and 
infrared (IR) regions of the electromagnetic spectrum. EarthSat employed a two-step 
process to orthorectify the TM imagery. The first step orthorectified TM scenes for which 
government-provided ground control points (GCPs) were available. The second step used 
the initially orthorectified scenes as control to tie the remaining raw imagery together 
using a single block adjustment. These orthorectified regions correspond to general land 
areas, or blocks, throughout the world. The result of this process is the block- 
orthorectified set of Landsat TM imagery that was delivered to NASA through the SDP. 
The SDP contract specification for horizontal geoposi tional accuracy of the TM dataset is 
a net root mean square error ( RMSE ne , ) of 50 meters. 

SSC ESA Directorate personnel then independently validated the orthorectified TM 
imagery for horizontal geometric accuracy. Additional validation-dedicated, government- 
provided ground control data was used to this end (i.e., the validation-dedicated GCPs 
were separate and independent of the GCPs used in the EarthSat orthorectification 
process). However, because of the limited number of additional government-provided 
GCPs within each scene, the validation assessment was performed by geographic block 
as opposed to individual scenes. Table 1 presents the results of the TM geolocational 
accuracy assessment including the x-component of the root mean square error ( RMSE X ), 
the y-component of the root mean square error (RMSEy), and the RMSE net . Additional 
background and test procedures used in the TM validation effort may be found in the 
publicly available TM validation reports located on the ESA Web site at 
http://www.esa.ssc.nasa.gOv/datapurchase/v v/es/es val reports.asp . 
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Table 1. TM Validation Results 


TM Block 


RMSE X i RMSEy RMSE net 

(meters) ! (meters) (meters) 

i 


Number of 
GCPs 
Available 


Number of 
GCPs 
Utilized 


1 Alaska 

30.57 

i 33.40 

45.28 

61 

l 43 

I Balkans 

j 23.17 

21.53 

31.62 

! 78 

i 49 

1 Caribbean 

1 19.17 

| 19.87 

27.61 

67 

i 53 

i Central America 

j 19.44 

t 15.32 

I 24.75 

12 

6 

; Central Asia 

18.45 

27.53 

j 33.14 

27 

18 

Central North America 

19.33 

18.56 

26.80 

57 

47 

East Africa 

20.88 

18.97 

28.21 

149 

135 

Eastern North America 

18.66 

18.90 

26.56 

62 

50 

Europe 

24.46 

26.64 

36.16 

140 

96 

Middle East 

32.84 

f 29.12 

43.89 

89 

56 

North Africa 

28.95 

i 38.28 

48.00 

6 

6 

Northeast Asia 

24.42 

25.61 

35.39 

80 

49 

Northern South America 

17.57 

39.36 

43.11 

92 

73 

Northwest Asia 

35.81 

27.39 

45.09 

12 

10 

South Africa 

17.86 

19.22 

26.24 

82 

50 

Southeast Asia 

24.38 

26.45 

35.97 

130 

90 

Southern South America 

1.58 

2.40 

2.87 

9 

9 

Western North America 

16.08 

14.45 

21.6 

73 

55 


NOTE: Highlighted data denotes blocks where fewer than 20 ground control points were available for 
statistical analysis. 


The ETM+ sensor onboard Landsat 7 is also a high-spatial-resolution (30-meter GSD) 
system with seven multispectral bands spanning the visible through IR spectral regions 
and an additional panchromatic band. The ETM+ imagery used in this assessment was 
also orthorectified by EarthSat. 


EarthSat employed a two-step process for ETM+ imagery orthorectification that is very 
similar to the TM orthorectification process. The first step exploited the TM scenes that 
had previously been orthorectified using government-provided GCPs as ground control 
for a relative (thin-plate spline) orthorectification of the corresponding ETM+ image. In 
the second step, the remaining raw ETM+ imagery was tied together using a block 
adjustment to yield near- world coverage of orthorectified ETM+ imagery. Exceptions to 
this orthorectification process were ETM+ geographic blocks identified as "Greenland,” 
"Indonesia,” “Islands,” “Upper South America,” and “Siberia.” In these cases, the 
horizontal control was provided for these regions through the ETM+ definitive 
ephemeris. 

SSC remote sensing personnel validated the orthorectified ETM+ imagery for horizontal 
geometric accuracy by performing both relative and absolute accuracy assessments. The 
relative accuracy assessment refers to the computation, examination, and analysis of 
statistical data representative of the relationship that exists between the ETM+ 
orthorectified imagery and the corresponding TM orthorectified imagery. Distinguishable 
geographic landmarks that were identified within both sets of imagery were used to 
perform this validation activity. The absolute accuracy assessment refers to the 


Page 3 of 9 






Geopositional Accuracy Validation of EarthSat GeoCover™ Orthorecti fled Landsat 
ETM+ Imagery 

computation, examination, and analysis of the statistical data representative of the 
relationship between ETM+ orthorectified imagery and government-provided GCPs. 

Geodetic Control Reference Data 


Two separate and independent sets of government-provided GCPs arc key to the 
orthorectification and validation processes. One of these sets of GCPs was delivered to 
EarthSat for use in the orthorectification of Landsat TM data. The second, independent 
set of GCPs was delivered to the ESE SDP program for use in the validation of Landsat 
TM and ETM+ geopositional accuracy. The control information was available in the 
form of both hardcopy documentation and softcopy computer files that duplicated the 
hardcopy coordinate documentation. The softcopy version enabled control point 
reference data to be imported into image processing software with minimal opportunities 
for error. The hardcopy documentation included pencil drawings of the terrain 
surrounding the geographic landmarks depicted by the GCPs. This data was essential to 
the Landsat TM geopositional accuracy assessment and to the Landsat ETM+ 
geopositional accuracy assessment. 

Approach 

To understand the ETM+ validation approach, it is useful to define the ETM+ horizontal 
accuracy specifications. The geometric accuracy specified for Landsat ETM+ is defined 
both in terms of relative accuracy to TM imagery and of absolute accuracy to ground 
coordinates. The relative geolocational accuracy specification for orthorectified ETM 4 - 
imagery procured through the SDP is an RMSE net of ±40 meters. Similarly, the absolute 
geolocational accuracy specification for Landsat orthorectified ETM+ imagery procured 
through the SDP is an RMSE net of ±64 meters. Both of these specifications are stipulated 
in NASA contract NAS 13-02032 with EarthSat. If the RMSE net values exceeded ±40 
meters for the relative assessment or ±64 meters for the absolute assessment, then the 
data was analyzed on a scene-by- scene basis to attempt to locate points that may have 
been erroneously biased because of incorrect analyst interpretation. When specific points 
were identified as having exceptionally large geolocational errors, the scenes were re- 
examined near the area in question. If the imagery was determined to be flawed, then the 
scene or scenes would be discussed with EarthSat for discrepancy resolution. 

To verify the relative geometric accuracy of the ETM+ imagery, selected ETM+ scenes 
were compared to the corresponding verified/validated TM scenes. The locations of 
identifiable features in ETM+ scenes were compared to the locations of the same features 
within the TM scenes. For each ETM+ scene, a population of 10 or more identifiable 
geographic landmarks were located and recorded in both the TM and the ETM± scenes. 
The resulting array of Universal Transverse Mercator (UTM) coordinates constituted the 
relative assessment dataset for the selected ETM± scene. The datasets for each scene 
were then compiled into a block dataset, and the statistics calculated from the data in this 
block dataset were compared to the relative geometric specifications for orthorectified 
ETM± data. 
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To verify the absolute accuracy of the ETM+ imagery, selected scenes were evaluated 
using the aforementioned government-provided GCPs. When possible, the geographic 
landmarks in the ETM- imagery that corresponded to the GCP hardcopy drawings were 
located. This process resulted in a dataset containing the imagery UTM coordinates and 
the corresponding government-provided reference LTM coordinates. As before, the 
arrays for each scene were then compiled into a block dataset, and the statistics calculated 
from the data in this block dataset were compared to the absolute geometric 
specifications for orthorectified ETM+ data. 

Dataset 


Two limiting factors determined the size of the datasets that were used for the ETM+ 
relative accuracy and absolute accuracy assessments. The first, most limiting factor was 
the small number of ETM+ images, compared to global coverage, that could be used for 
this exercise. The small number of validation scenes is directly related to the small 
number of government-provided GCP data packets that provided the validation truth data 
for the TM geolocational accuracy assessment and for the ETM.+ absolute accuracy 
assessment. An illustration of the global distribution of validation scenes is available in 
Appendix A. The second limiting factor was the number of definitive monuments (GCPs 
in the absolute assessment and geographic landmarks in the relative assessment) per 
scene that could be selected to perform the necessary computations and analyses. A set of 
1 00 images was used for both the ETM+ relative accuracy assessment and the ETM+ 
absolute accuracy assessment. 

Procedure 


The relative horizontal geometric accuracy assessment of ETM+ imagery began with the 
identification of images that corresponded to previously validated TM images. Next, the 
data was analyzed using the GCP editor function available in ERDAS IMAGINE image 
processing software. For each scene, the Landsat ETM+ and corresponding TM images 
were loaded into separate viewers. Once loaded, both the ETM+ and the TM images were 
inspected for clearly identifiable features. When specific landmarks, such as rock 
formations, roads, and waterways, were identifiable in both scenes, these landmarks were 
selected and geographically located. This action populated the X-Input and Y -Input fields 
in the software’s GCP editor. For each ETM+ scene, a population of 10 or more 
identifiable geographic landmarks were located and recorded in both the TM and the 
ETM+ scenes. The resulting array of UTM coordinates constituted the relative 
assessment dataset for the selected ETM+ scenes. 

The ETM+ absolute horizontal geometric accuracy assessment began with the 
identification of images that corresponded to the government-provided GCP packets. 
This assessment was also performed using the GCP editor function available in ERDAS 
IMAGINE. For each scene, the same ETM+ image was opened in both viewers. Once 
opened, the government-provided GCPs were imported into the GCP editor function as 
reference points. This populated one of the viewers with crosshairs denoting the locations 
of the provided GCPs. At this point, the monuments called out in the GCP hardcopy data 
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packets were located in the viewer that was not displaying the government-provided 
GCPs. Once located, the monuments were selected, populating the GCP editor function 
as input points. The resulting array of UTM coordinates constituted the absolute 
assessment dataset for the selected ETM+ scene. Typically all points associated with this 
data were not located. The most common reason for inability to locate the GCPs was that 
the control locations were not within the confines of the scene being examined. Other 
reasons for not locating GCP landmarks included terrain uniformity and land change. 

In both the ETM+ relative and absolute accuracy assessments, the individual scene 
datasets were exported and compiled into larger datasets corresponding to the geographic 
block in which the imagery was originally orthorecti fied. These larger datasets contained 
the point IDs, input coordinates, and reference coordinates for each scene and provided 
the required input into the ESA Directorate's Visual Basic program written to compute 
relative geopositional accuracy. For each point, the Visual Basic program computed the 
X-differences and Y-differences, the squares of the X-differences and Y-differences, the 
RMSE X and RMSE y , and the RMSE nel using the following accuracy formulas: 


RMSE X 

RMSE y 

RMSE net 



= ^{RMSE x f + ( RMSE V ) 2 


where X inpm , Y input are the coordinates of the ETM+ input points; X con troi, Y control are the 
coordinates of the TM reference points; and n is the total number of usable points. 

Results 


The relative assessment used all 100 ETM+ scenes located throughout the world. From 
these scenes, 1065 geographic landmarks were used for analysis. All 18 blocks passed the 
40-meter RMSE net relative geographic accuracy specification. The statistics calculated on 
the ETM+ relative assessment are presented in Table 2. The maximum RMSE net found for 
the relative assessment was 34.90 meters, the minimum RMSE nel found for the relative 
assessment was 13.89 meters, and the average RMSE nel found for the relative assessment 
was 25.84 meters. 
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Table 2. ETM+ Relative Horizontal Geometric Accuracy Calculations 


ETM+ Geographic Block 

I RMSE X 

(meters) 

I RMSE y 

(meters) 

1 RMSE net 
(meters) 

! Number of 
Point Pairs 

i Africa - World Summit on 

Sustainable Development 

23.29 

25.15 

34.28 

1 

Caribbean 

15.61 

19.37 

24.88 

30 ! 

Europe 

20.50 

18.15 

27.38 

70 

| Europe 2 

17.95 

17.67 

25.19 

60 | 

Indonesia 

23.63 

25.38 

34.68 

25 

Lower North America 

16.97 

17.35 

24.26 

140 

Lower South America 

26.21 

18.73 

32.22 

20 

NE Africa 

17.19 

19.40 

25.92 

100 

NE Asia 

16.46 

14.87 

22.18 

40 

NW Africa 

22.12 

27.00 

34.90 

1 20 

NW Asia 

12.68 

13.51 

18.53 

30 

SE Asia 

25.62 

18.78 

31.76 

130 

South Africa 

14.04 

14.27 

20.02 

80 ^ 

SW Asia 

20.60 

20.99 

29.41 

50 

SW Asia 2 

14.41 

14.78 

20.64 

30 | 

Upper North America 2 

10.14 

9.49 

13.89 

20 

Upper North America 3 (Alaska) 

18.10 

19.70 

26.76 

110 

Upper South America 

12.84 

12.88 

18.19 

80 


The absolute assessment also used all 100 ETM+ scenes located throughout the world. 
From these scenes, 750 government -provided control points were used in the analysis. 
Table 3 presents the results of the ETM+ absolute geometric assessment. Note that the 
geographic blocks identified as “Africa — World Summit on Sustainable Development,' 
“Indonesia,” “Lower South America,” “Northwest Africa,” and “Upper North America 
2” contained fewer than the 20 points suggested by the Federal Geographic Data 
Committee (FGDC) for statistical analysis. The statistics for these four blocks were 
nonetheless calculated and included in Table 3. All 18 of the blocks passed the 64-meter 
RMSE ne t absolute accuracy specification. The maximum RMSE net found for the absolute 
assessment was 51.92 meters, the minimum RMSE net found for the absolute assessment 
was 22.98 meters, and the average RMSE net found for the absolute assessment was 34.88 
meters. 
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Table 3. ETM+ Absolute Horizontal Geometric Accuracy Calculations 


ETM+ Geographic Block 

RMSE X 

(meters) 

RMSEy 

(meters) 

! RMSE net 
(meters) 

i 

Number of 
GCPs 
Used 

Africa - World Summit on 
Sustainable Development 

| 43.37 

[ 28.54 

| 51.92 

10 

Caribbean 

17.39 

27.26 

32.33 

35 

Europe 

20.04 

24.99 

32.04 

42 

i Europe 2 

18.88 

25.05' 

31.37 

39 

I Indonesia 

21.63 

20.60 

29.87 

17 

Lower North America 

23.08 

24.23 

33.46 

96 

Lower South America 

13.17 

19.20 

23.29 

8 

NE Africa 

15.65 

16.83 

22.98 

| 60 

I NE Asia 

17.43 

23.12 

28.95 

! 25 

NW Africa 

17.87 

35.17 

39.45 

i 3 

1 NW Asia 

22.52 

23.23 

32.36 

20 

SE Asia 

23.42 

24.28 

33.73 

119 

South Africa 

18.55 

24.30 

30.57 

78 

SW Asia 

30.76 

34.62 

46.31 

57 

SW Asia 2 1 

26.75 

37.81 

46.31 

30 

Upper North America 2 

15.73 

23.22 

28.04 

7 

Upper North America 3 (Alaska) 

18.80 

28.70 

34.31 

45 

Upper South America 

34.37 

37.04 

50.53 

59 


NOTE: Highlighted data denotes blocks where fewer than 20 ground control points were available for 


statistical analysis. 

Limitations 


Because the validated Landsat TM imagery was treated as a “truth” dataset for the 
relative accuracy assessment of the ETM+ imagery, any errors that might exist in the 
validated TM imagery could potentially have adverse effects on the relative accuracy 
assessment of the ETM+ imagery. 

The ETM+ geolocational accuracy assessment approach was dependent on the 
geographic distribution of the available government-provided GCPs used in these 
assessments. This scene distribution is shown in Appendix A. 


Inconsistencies could arise during the absolute accuracy assessment as a result of 
differences in the interpretations of the government-provided GCP drawings and 
imagery. The hand-drawn target area descriptions also reflect subjectivity; they are based 
on the perception of the artist, which may differ more or less from that of the analyst. 

Finally, the possibility exists for an inherent bias specific to the analyst as a result of the 
variability in image interpretation skills. The final selection of points also reflects analyst 
subjectivity. 
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